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Restoring  turbulence-degraded 
underwater  images 


Andrey  Kanaev  and  Weilin  Hou 


Modifications  to  atmospheric  imaging  techniques  can  improve  the 
resolution  of  images  taken  under  water. 

Underwater  imaging  is  essential  for  a  variety  of  scientific  and 
military  applications,  such  as  optical  communications,  situa¬ 
tional  awareness,  diver  visibility,  and  mine  detection.  Unfortu¬ 
nately,  underwater  imaging  is  severely  impaired  by  scattering 
and  optical  turbulence  associated  with  refraction  index  fluctu¬ 
ations.  Because  of  the  similarities  between  underwater  and  at¬ 
mospheric  image  degradation,  approaches  taken  to  solve  the 
underwater  image  restoration  problem  originate  in  atmospheric 
turbulence  compensation  algorithms.1'2  However,  applying 
unmodified  atmospheric  techniques  to  underwater  images  fails 
because  of  the  differences  in  the  scale  of  the  refraction  index 
fluctuations. 

Scattering  may  come  from  the  water  itself,  particulates  sus¬ 
pended  in  the  water  column,  or  turbulence  present  along  the 
propagation  path.  Particulates  contribute  more  significantly 
to  image  degradation  in  general,  especially  at  larger  viewing 
angles.  Near-forward  turbulent  scattering  dominates  near  the 
surface,  mixed  layer,  and  bottom  of  the  water  column.  As  a 
result  of  large  index  of  refraction  fluctuations,  underwater 
imaging  is  subject  to  stronger  degradation  than  atmospheric 
imaging.  Examples  of  degradation  include  significant  loss  of 
signal,  time-dependent  blur,  noise,  and  feature  motion  or  dis¬ 
tortion.  The  multitude  of  techniques  designed  to  image  through 
atmospheric  turbulence  has  led  to  advances  in  imaging  through 
water  turbulence  and  allowed  us  to  develop  a  multi-frame 
underwater  restoration  algorithm.3 

Our  algorithm,  based  on  the  Tucky  patch'  atmospheric  com¬ 
pensation  technique,  uses  a  sequence  of  consecutive  short- 
exposure  underwater  images  as  input.  We  begin  by  denoising 
each  frame  in  the  sequence  to  avoid  noise  amplification  during 
further  processing.  Next,  we  estimate  the  optical  flow  between 
the  reconstructed  image  and  the  current  frame.  Simultaneously, 
we  compute  an  uncertainty  measure  of  the  optical  flow  estima¬ 
tion  to  control  for  possible  errors  in  this  step.4  We  then  use  this 


Figure  1.  A  typical  image  of  the  US  Air  Force  1951  resolution  chart 
taken  underwater  at  Lake  Skaneateles. 

optical  flow  estimate  for  motion  compensation  by  warping  the 
current  frame.  Fourth,  we  compute  a  gradient-based  image  qual¬ 
ity  for  the  current  and  reconstructed  frame.  We  then  calculate 
local  gain  coefficients  based  on  the  difference  between  the  com¬ 
puted  image  quality  maps.  Finally,  we  apply  a  lucky  patch  fu¬ 
sion  process  using  the  current  frame,  the  reconstructed  frame, 
and  the  local  gain  coefficients.2  The  fusion  is  not  performed  in 
areas  of  the  image  where  the  optical  flow  uncertainty  measure 
exceeds  a  certain  threshold. 

We  tested  our  algorithm  on  data  collected  in  July  2010  at  Lake 
Skaneateles,  where  we  imaged  a  target  over  a  5m  path  length 
at  various  depths  in  the  water  column.  Figure  1  shows  a  typical 
image  of  the  US  Air  Force  1951  resolution  chart  collected  with 
a  monochromatic  electro-optical  camera  during  the  exercise. 
Figure  2  compares  a  10-frame  mean  image  with  the  restored  im¬ 
age  displaying  the  resolution  advantage  of  at  least  two  vertical 
line  bands. 
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Additionally,  we  collected  laboratory  imagery  in  a  convective 
tank  that  was  designed  and  implemented  at  the  Oceanography 
Division  of  the  Naval  Research  Laboratory  at  Stennis  Space  Cen¬ 
ter.  The  tank  houses  highly  conductive  stainless-steel  heat  dis¬ 
sipation  plates  on  the  top  and  bottom.  Temperature-controlled 
water  is  used  to  regulate  the  plate  temperature  and,  thus,  the 
intensity  of  the  turbulent  structure  within  the  tank.  Under¬ 
water  imagery  from  the  laboratory  tank,  collected  with  the  same 
camera  used  in  the  field  exercise,  yields  a  simple  pattern  of  hor¬ 
izontal  lines  with  spatial  frequencies  of  1250  and  2500  cycles  per 
radian  (see  Figure  3).  Figure  4  compares  a  mean  image  to  the  re¬ 
stored  image.  The  restored  image  shows  better  definition  in  the 
high-frequency  pattern  and  sharper  low-frequency  lines. 

In  summary,  we  developed  and  tested  an  algorithm  to  re¬ 
store  degraded  underwater  imagery  obtained  in  the  field  and  in 


Figure  2.  Left:  A  mean  image  from  10  underwater  exposures  at  Lake 
Skaneateles.  Right:  The  same  image  restored  using  the  'lucky  patch' 
algorithm  with  optical  flow  uncertainty  measures. 


Figure  3.  A  typical  underwater  image  taken  in  the  laboratory. 


Figure  4.  Left:  A  mean  laboratory  image  from  40  short  exposures. 
Right:  The  same  image  restored  using  the  lucky  patch  algorithm  with 
optical  flow  uncertainty  measures. 

laboratory  conditions.  Our  approach  uses  a  sequence  of  short 
exposure  frames  to  obtain  a  single  higher-resolution  image.  The 
algorithm  is  based  on  the  lucky  patch  atmospheric  compensa¬ 
tion  technique  modified  with  nonlinear  gain,  nonlocal  filtering, 
and  optical  flow  motion  compensation  with  uncertainty  mea¬ 
sure  control.  For  future  work,  we  plan  to  use  and  modify  even 
more  powerful  atmospheric  imaging  techniques  like  adaptive 
optics  in  a  challenging  underwater  environment. 

This  work  is  sponsored  by  Office  of  Naval  Research/Naval  Research 
Laboratory  program  element  627 82N. 
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